Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix panic on metricbeat test modules #18797

Merged
merged 5 commits into from
May 29, 2020

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented May 27, 2020

What does this PR do?

Fix panic on `metricbeat test modules`

Since metricbeat light modules support processors (#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issue is also added.

Why is it important?

metricbeat test modules is not working with modules defined in the main configuration file.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Related issues

Since metricbeat light modules support processors (elastic#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issue is also added.
@jsoriano jsoriano added bug review needs_backport PR is waiting to be backported to other branches. [zube]: In Review Team:Integrations Label for the Integrations team v7.8.0 v7.7.1 labels May 27, 2020
@jsoriano jsoriano requested a review from mtojek May 27, 2020 20:23
@jsoriano jsoriano self-assigned this May 27, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels May 27, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 27, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [jsoriano commented: jenkins run the tests please]

  • Start Time: 2020-05-29T10:23:00.995+0000

  • Duration: 75 min 45 sec

Test stats 🧪

Test Results
Failed 0
Passed 4339
Skipped 971
Total 5310

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Note the conflict in the changelong.

metricbeat/cmd/test/modules.go Show resolved Hide resolved
@jsoriano
Copy link
Member Author

@ChrsMark conflict resolved and comment added, though the type is not public. Could you take a last look?

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment was for the SetACKHandler but somehow GH showed it for the one above :).

/lgtm

@jsoriano
Copy link
Member Author

jenkins run the tests please

@jsoriano jsoriano merged commit 25b8bf1 into elastic:master May 29, 2020
@jsoriano jsoriano deleted the metricbeat-test-modules-panic branch May 29, 2020 14:37
@jsoriano jsoriano removed the needs_backport PR is waiting to be backported to other branches. label May 29, 2020
jsoriano added a commit to jsoriano/beats that referenced this pull request May 29, 2020
Since metricbeat light modules support processors (elastic#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 25b8bf1)
jsoriano added a commit to jsoriano/beats that referenced this pull request May 29, 2020
Since metricbeat light modules support processors (elastic#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 25b8bf1)
jsoriano added a commit to jsoriano/beats that referenced this pull request May 29, 2020
Since metricbeat light modules support processors (elastic#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 25b8bf1)
v1v added a commit to v1v/beats that referenced this pull request Jun 2, 2020
…-stage-level

* upstream/master: (30 commits)
  Add a GRPC listener service for Agent (elastic#18827)
  Disable host.* fields by default for iptables module (elastic#18756)
  [WIP] Clarify capabilities of the Filebeat auditd module (elastic#17068)
  fix: rename file and remove extra separator (elastic#18881)
  ci: enable JJBB (elastic#18812)
  Disable host.* fields by default for Checkpoint module (elastic#18754)
  Disable host.* fields by default for Cisco module (elastic#18753)
  Update latest.yml testing env to 7.7.0 (elastic#18535)
  Upgrade k8s.io/client-go and k8s keystore tests (elastic#18817)
  Add missing Jenkins stages for Auditbeat (elastic#18835)
  [Elastic Log Driver] Create a config shim between libbeat and the user (elastic#18605)
  Use indexers and matchers in config when defaults are enabled (elastic#18818)
  Fix panic on `metricbeat test modules` (elastic#18797)
  [CI] Fix permissions in MacOSX agents (elastic#18847)
  [Ingest Manager] When not port are specified and the https is used fallback to 443 (elastic#18844)
  [Ingest Manager] Fix install service script for windows (elastic#18814)
  [Metricbeat] Fix getting compute instance metadata with partial zone/region config (elastic#18757)
  Improve error messages in s3 input (elastic#18824)
  Add memory metrics into compute googlecloud (elastic#18802)
  include bucket name when logging error (elastic#18679)
  ...
jsoriano added a commit that referenced this pull request Jun 3, 2020
Since metricbeat light modules support processors (#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 25b8bf1)
jsoriano added a commit that referenced this pull request Jun 3, 2020
Since metricbeat light modules support processors (#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 25b8bf1)
jsoriano added a commit that referenced this pull request Jun 4, 2020
…8853)

Since metricbeat light modules support processors (#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 25b8bf1)

Add nil pipeline from #16715, required for the fix.

(cherry picked from commit 7a1b524)

Co-authored-by: Steffen Siering <[email protected]>
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…es` (elastic#18853)

Since metricbeat light modules support processors (elastic#15923), module
initialization requires a publisher in the beat so modules can attach
their processors. `metricbeat test modules` is not initializing as
normal metricbeat commands, and it is not initializing any output or
publisher pipeline, so metricbeat panics when trying to initialize
modules with the new method.

This change adds a dummy publisher for this case, and fixes also a
condition that was adding a `nil` module option, causing additional
panics. A test that reproduced the issues is also added.

(cherry picked from commit 316793d)

Add nil pipeline from elastic#16715, required for the fix.

(cherry picked from commit 257fdfc)

Co-authored-by: Steffen Siering <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

metricbeat test modules panics
4 participants